Given a set of calibrated images of a scene, we present an approach that produces a simple, compact, and actionable 3D world representation by means of 3D primitives. While many approaches focus on recovering high-fidelity 3D scenes, we focus on parsing a scene into mid-level 3D representations made of a small set of textured primitives. Such representations are interpretable, easy to manipulate and suited for physics-based simulations. Moreover, unlike existing primitive decomposition methods that rely on 3D input data, our approach operates directly on images through differentiable rendering.

We present a simulation framework for lunar construction work involving multiple autonomous machines. The framework supports modelling of construction scenarios and autonomy solutions, execution of the scenarios in simulation, and analysis of work time and energy consumption throughout the construction project. The simulations are based on physics-based models for contacting multibody dynamics and deformable terrain, including vehicle-soil interaction forces and soil flow in real time. A behaviour tree manages the operational logic and error handling, which enables the representation of complex behaviours through a discrete set of simpler tasks in a modular hierarchical structure. High-level decision-making is separated from lower-level control algorithms, with the two connected via ROS2. Excavation movements are controlled through inverse kinematics and tracking controllers. The framework is tested and demonstrated on two different lunar construction scenarios that involve an excavator and dump truck with actively controlled articulated crawlers.

We apply the physics-learning duality to molecular systems by complementing the physical description of interacting particles with a dual learning description, where each particle is modeled as an agent minimizing a loss function. In the traditional physics framework, the equations of motion are derived from the Lagrangian function, while in the learning framework, the same equations emerge from learning dynamics driven by the agent loss function. The loss function depends on scalar quantities that describe invariant properties of all other agents or particles. To demonstrate this approach, we first infer the loss functions of oxygen and hydrogen directly from a dataset generated by the CP2K physics-based simulation of water molecules. We then employ the loss functions to develop a learning-based simulation of water molecules, which achieves comparable accuracy while being significantly more computationally efficient than standard physics-based simulations.

It has been shown recently that physics-based simulation significantly enhances the disassembly capabilities of real-world assemblies with diverse 3D shapes and stringent motion constraints. However, the efficiency suffers when tackling intricate disassembly tasks that require numerous simulations and increased simulation time. In this work, we propose a State-Based Disassembly Planning (SBDP) approach, prioritizing physics-based simulation with translational motion over rotational motion to facilitate autonomy, reducing dependency on human input, while storing intermediate motion states to improve search scalability. We introduce two novel evaluation functions derived from new Directional Blocking Graphs (DBGs) enriched with state information to scale up the search. Our experiments show that SBDP with new evaluation functions and DBGs constraints outperforms the state-of-the-art in disassembly planning in terms of success rate and computational efficiency over benchmark datasets consisting of thousands of physically valid industrial assemblies.

Continuum robots, which often rely on interdisciplinary and multimedia collaborations, have been increasingly recognized for their potential to revolutionize the field of human-computer interaction (HCI) in varied applications due to their adaptive, responsive, and flexible characteristics. Despite their promises, the lack of an integrated framework poses a significant limitation for both users and developers, resulting in inefficiency and complexity during preliminary developments. Thus, this paper introduces a unified framework for continuum robotic systems that addresses these challenges by integrating system architecture, dynamics computation, and control strategy within a computer-aided design (CAD) platform. The proposed method allows for efficient modeling and quick preview of the robot performance, and thus facilitating iterative design and implementation, with a view to enhancing the quality of robot developments.

From chemicals to energy, artificial intelligence (AI) is already showing just how far it can help achieve global sustainability targets across different industrial sectors. One example is Petroliam Nasional Berhad (PETRONAS), has committed to achieving net-zero carbon emissions by 2050. For the Malaysian oil and gas multinational, plant reliability is key to achieving its sustainability goals. PETRONAS identified that early insight into impending equipment failure would enable plant operators to fix equipment proactively before small issues become bigger problems. Proof of concept came via a pilot project in their corporate cloud on Microsoft Azure at four upstream and two downstream units.

Physics-based simulations, that staple of traditional HPC, may be evolving toward an emerging, AI-based technique that could radically accelerate simulation runs while cutting costs. Called "surrogate machine learning models," the topic was a focal point in a keynote on Tuesday at the International Conference on Parallel Processing by Argonne National Lab's Rick Stevens. Stevens, ANL's associate laboratory director for computing, environment and life sciences, said early work in "surrogates," as the technique is called, shows tens of thousands of times (and more) speed-ups and could "potentially replace simulations." In his keynote, entitled, "Exascale and Then What?: The Next Decade for HPC and AI," Stevens explained surrogates this way: "You have a system, it could be a molecular system or drug design…, and you have a physics-based simulation of it… You run this code and capture the input-output relationships of the core simulation… You use that training data to build an approximate model. These are typically done with neural networks… and this surrogate model approximates the simulation, and typically it is much faster. Of course, it has some errors, so then you use that surrogate model to search the space, or to advance time steps. And then maybe you do a correction step later."

Physics-based simulations typically operate with a combination of complex differentiable equations and many scientific and geometric inputs. Our work involves gathering data from those simulations and seeing how well tree-based machine learning methods can emulate desired outputs without "knowing" the complex backing involved in the simulations. The selected physics-based simulations included Navier-Stokes, stress analysis, and electromagnetic field lines to benchmark performance as numerical and statistical algorithms. We specifically focus on predicting specific spatial-temporal data between two simulation outputs and increasing spatial resolution to generalize the physics predictions to finer test grids without the computational costs of repeating the numerical calculation.

Since the earliest days of robotics, researchers have sought to build embodied agents to perform a variety of jobs, such as assistive tasks in factories [Oliff et al., 2020] or wildfire surveillance [Julian and Kochenderfer, 2019]. Following tremendous advancements in deep learning and convolutional neural networks in the past decade, researchers have been able to develop reinforcement learning (RL)-based embodied agents that interact with the real world on the basis of sensory observations. Software platforms such as OpenAI Gym [Brockman et al., 2016], Unity ML-Agents Toolkit [Juliani et al., 2018], and AI Habitat [Savva et al., 2019] have emerged to address the community's need for training and evaluating RL-based embodied agents end-to-end. Our research group was particularly intrigued by the AI Habitat platform, which offers a high-performance, photorealistic simulator, access to a sizeable library of visually-rich scanned 3D environments, and a modular software design. However, even though these platforms allow roboticists to reuse existing RL algorithms and train agents in simulators with ease, there is a critical step to using them for embodied agents which is only partially addressed: Connecting the trained agent with a real robot. Ideally, after training an RL agent in simulation one would like to take advantage of the extensive set of tools and knowledge from the robotics community to make it easy to embody that agent. One particularly popular tool from the robotics community is ROS, a robotics-focused middleware platform with extensive support for classical robotic mapping, planning and control algorithms ([mov, dwa]) as well as drivers for a wide variety of compute, sensing and actuation hardware. But ROS support for directly training an RL agent is limited, and Gazebo-- the standard simulation environment used for ROS systems-- cannot match the level of photorealism or simulation speed of tools specifically designed to train large-scale RL agents [Liang et al., 2019].